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CLAIMS FOR FILING IN THK Ufi ; 

1. Data defining a phoneme and word lattice, the data 
comprising: 

data for defining a plurality of nodes within the 
lattice and a plurality of links connecting the nodes 
within the lattice; 

data associating a plurality of phonemes with a 
respective plurality of links; and 

data associating at least one word with at least one 
of said links. 

2. Data according to any preceding claim, wherein said 
data defining said phoneme and word lattice is arranged 
in blocks of nodes. 

3. Data according to claim 1, further comprising data 
defining time stamp information for each of said nodes. 

4. Data according to claim 3, arranged in blocks of 
equal time duration. 

5. Data according to claim 2, further comprising data 
defining each blocks location within said database. 

6. Data according to claim 3, wherein said data 
defining a phoneme and word lattice is associated with 
further data defining a time sequential signal, and 
wherein said time stamp information is time synchronised 
with said time sequential signal. 

7. Data according to claim 6, wherein said further data 
defines an audio and/or video signal. 

8. Data according to claim 7, wherein said further data 
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defines at least speech data and wherein said data 
defining said phoneme and word lattice is derived from 
said further data. 

9. Data according to claim 8, wherein said speech data 
comprises audio data and wherein said data defining said 
phoneme and word lattice is derived by passing said audio 
signal through an automatic speech recognition system. 

10. Data according to claim 8, wherein said speech data 
defines the parol of a plurality of speakers, and wherein 
said data defines a separate phoneme and word lattice for 
the parol of each speaker. 

11. Data according to claim 1, further comprising data 
defining a weighting for the phonemes and/or words 
associated with said links. 

12. Data according to claim 1, wherein at least one of 
said nodes is connected to a plurality of other nodes by 
a plurality of links. 

13. Data according to claim 12, wherein at least one of 
said plurality of links connecting said node to said 
plurality of other nodes is associated with a phoneme and 
wherein at least one of said links connecting said node 
to said plurality of other nodes is associated with a 
word. 

14. A method of searching a database comprising data 
according to any preceding claim, in response to an input 
guery, the method comprising the steps of: 

generating phoneme data and/or word data 
corresponding to the input query; 

searching the phoneme and word lattice using the 



phoneme and/or word data generated for the input query; 
and 

outputting search results in dependence upon the 
results of said searching step. 

15. A method according to claim 14, wherein said 
searching step comprises the steps of: 

(i) searching the phoneme and word lattice using the 
word data generated for the user ' s input query to 
identify similar words within the phoneme and word 
lattice; 

(ii) selecting one or more portions of the phoneme 
and word lattice for further searching in response to the 
results of said word search; and 

(iii) searching said one or more selected portions 
of the phoneme and word lattice using the phoneme data 
generated for the user's input query. 

16. A method according to claim 15, wherein the results 
of the word search are output to the user before the 
phoneme search is performed on the selected portions of 
the database. 

17. A method according to claim 16, wherein said phoneme 
search is only performed in response to a further input 
by the user in response to the outputting of the results 
from the word search. 

18. A method according to claim 15, wherein said phoneme 
search is carried out by identifying a number of features 
within the phoneme sequence corresponding to the user's 
input query and identifying similar features within the 
data defining said phoneme lattice within the database. 



A method according to claim 18, wherein each of said 



features represents a unique sequence of phonemes within 
the phoneme data of the user's input query. 

20. A method according to claim 19, wherein said phoneme 
search employs a cosine measure to indicate the 
similarity between the phoneme data corresponding to the 
user's input query and the phoneme data within the 
database. 

21. A method according to claim 14, wherein said search 
results are output to a display. 

22. A method according to claim 14, wherein said input 
query by the user is input by voice, and wherein said 
step of generating phoneme data and word data employs an 
automatic speech recognition system. 

23. A method according to claim 14, wherein said input 
query is a typed input and wherein said step of 
generating phoneme data and word data employs a text-to- 
phoneme converter. 

24. An apparatus for searching a database comprising 
data according to claim 1, in response to an input query, 
the apparatus comprising: 

means for generating phoneme data and/or word data 
corresponding to the input query; 

means for searching the phoneme and word lattice 
using the phoneme and/or word data generated for the 
input query; and 

means for outputting search results in dependence 
upon the output from said searching means. 

25. An apparatus according to claim 24, wherein said 
searching means comprises: 
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(i) means for searching the phoneme and word lattice 
using the word data generated for the user's input query 
to identify similar words within the phoneme and word 
lattice; 

(ii) means for selecting one or more portions of the 
phoneme and word lattice for further searching in 
response to the results of said word search; and 

(iii) means for searching said one or more selected 
portions of the phoneme and word lattice using the 
phoneme data generated for the user's input query. 

26. An apparatus according to claim 25, wherein said 
output means is operable to output the results of the 
word search to the user before the phoneme search is 
performed on the selected portions of the database. 

27. An apparatus according to claim 26, wherein said 
phoneme search is only performed in response to a further 
input by the user in response to the outputting of the 
results from the word search. 

28. An apparatus according to claims 25, wherein said 
phoneme search is carried out by identifying a number of 
features within the phoneme sequence corresponding to the 
user's input query and identifying similar features 
within the data defining said phoneme lattice within the 
database. 

29. An apparatus according to claim 28, wherein each of 
said features represents a unique sequence of phonemes 
within the phoneme data of the user's input query. 

30. An apparatus according to claim 29, wherein said 
phoneme search employs a cosine measure to indicate the 
similarity between the phoneme data corresponding to the 



user's input query and the phoneme data within the 
database . 

31. An apparatus according to claim 24, wherein said 
output means comprises a display. 

32. An apparatus according to claim 24, wherein said 
input query by the user is a voice query, and wherein 
said means for generating phoneme data and word data 
comprises an automatic speech recognition system which is 
operable to generate said phoneme data and a word decoder 
which is operable to generate said word data. 

33. An apparatus according to claim 24, wherein said 
input query is a typed query and wherein said means for 
generating phoneme data and word data comprises a text- 
to-phoneme converter which is operable to generate said 
phoneme data. 

34. An apparatus for generating annotation data for use 
in annotating a data file comprising audio data, the 
apparatus comprising: 

an automatic speech recognition system for 
generating phoneme data for audio data in the data file; 

a word decoder for identifying possible words within 
the phoneme data generated by the automatic speech 
recognition system; and 

generating means for generating annotation data by 
combining the generated phoneme data and the decoded 
words . 

35. An apparatus for generating annotation data for use 
in annotating a data file comprising text data, the 
apparatus comprising: 

a text to phoneme converter for generating phoneme 
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data for text data in the data file; and 

generating means for generating annotation data by 
combining the phoneme data and words in the text data. 

36. An apparatus for generating annotation data for use 
in annotating a data file, the apparatus comprising: 

input means for receiving an input voice signal; 

speech recognition means for converting the input 
voice signal into phoneme data and words; and 

generating means for generating annotation data by 
combining the phoneme data and the words . 

37. An apparatus for generating annotation data for use 
in annotating a data file, the apparatus comprising: 

input means for receiving a typed input from a user; 

converting means for converting words in the typed 
input into phoneme data; and 

generating means for generating annotation data by 
combining the phoneme data and words in the typed input. 

38. An apparatus for generating annotation data for use 
in annotating a data file, the apparatus comprising: 

means for receiving image data representative of 

text ; 

character recognition means for converting said 
image data into text data; 

converting means for converting words in the text 
data into phoneme data; and 

generating means for generating annotation data by 
combining the phoneme data and words in the text data. 

39. An apparatus according to claim 34, wherein said 
annotation data defines a phoneme and word lattice and 
wherein said generating means comprises: 

(i) means for generating data defining a plurality 
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of nodes within the lattice and a plurality of links 
connecting the nodes within the lattice; 

(ii) means for generating data associating a 
plurality of phonemes of the phoneme data with a 
respective plurality of links; and 

(iii) means for generating data associating at least 
one of the words with at least one of said links. 

40. An apparatus according to claim 39, wherein said 
generating means is operable to generate said data 
defining said phoneme and word lattice in blocks of said 
nodes . 

41. An apparatus according to claim 39, wherein said 
generating means is operable to generate data defining 
time stamp information for each of said nodes . 

42. An apparatus according to claim 41, wherein said 
generating means is arranged to generate said phoneme and 
word lattice data in blocks of equal time duration. 

43. An apparatus according to claim 40, wherein said 
generating means is operable to generate data which 
defines each block's location within a database. 

44. An apparatus according to claim 41, wherein said 
data file includes a time sequential signal, and wherein 
said generating means is operable to generate time stamp 
data which is time synchronised with said time sequential 
signal . 

45. An apparatus according to claim 44, wherein said 
time sequential signal is an audio and/or video signal. 



An apparatus according to claim 34, wherein said 



audio data includes audio data which defines the parol of 
a plurality of speakers, and wherein said generating 
means is operable to generate data which defines separate 
phoneme and word annotation data for the parol of each 
speaker. 

47- An apparatus according to claim 35, wherein said 
text data defines the parol of a plurality of speakers, 
and wherein said generating means is operable to generate 
data defining separate phoneme and word annotation data 
for the parol of each speaker. 

48. An apparatus according to claim 34, wherein said 
speech recognition system is operable to generate data 
defining a weighting for the phonemes in the phoneme 
data. 

49. An apparatus according to claim 34, wherein said 
word decoder is operable to generate data defining a 
weighting for the words identified within said phoneme 
data. 

50. An apparatus according to claim 39, wherein said 
means for generating data defining a plurality of nodes 
and a plurality of links is operable to define at least 
one node which is connected to a plurality of other nodes 
by a plurality of links. 

51. An apparatus according to claim 50, wherein at least 
one of said plurality of links connecting said node to 
said plurality of other nodes is associated with a 
phoneme and wherein at least one of said links connecting 
said node to said plurality of other nodes is associated 
with a word. 



52. An apparatus according to claim 36, wherein said 
speech recognition means is operable to generate data 
defining a weighting for the phonemes in the phoneme 
data. 

53. An apparatus according to claim 52, wherein said 
speech recognition means is operable to generate data 
defining a weighting for the words within the word data. 

54. An apparatus according to claim 36, further 
comprising means for associating said annotation data 
with said data file. 

55. An apparatus according to claim 37, wherein said 
converting means comprises an automatic phonetic 
transcription unit which generates said phoneme data from 
words within the typed input. 

56. An apparatus according to claim 38, wherein said 
converting means comprises an automatic phonetic 
transcription unit which generates said phoneme data from 
words within the text data output by said character 
recognition means. 

57. An apparatus according to claim 38, further 
comprising means for associating said annotation data 
with either said image data representative of said text 
or with said text data. 

58. An apparatus according to claim 38, wherein said 
receiving means comprises a document scanner or a 
facsimile machine. 

59. A method of generating annotation data for use in 
annotating a data file comprising audio data, the method 
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comprising the steps of: 

using an automatic speech recognition system to 
generate phoneme data for audio data in the data file; 

using a word decoder to identify possible words 
within the phoneme data generated by the automatic speech 
recognition system; and 

generating annotation data by combining the 
generated phoneme data and the decoded words. 

60. A method of generating annotation data for use in 
annotating a data file comprising text data, the method 
comprising the steps of: 

using a text to phoneme converter to generate 
phoneme data for text data in the data file; and 

generating annotation data by combining the phoneme 
data and words in the text data. 

61. A method of generating annotation data for use in 
annotating a data file, the method comprising the steps 
of: 

receiving an input voice signal; 

processing the input voice signal using a speech 
recognition system to generate phoneme data and word data 
for the input voice signal; and 

generating annotation data by combining the phoneme 
data and the word data generated for the input voice 
signal . 

62. A method of generating annotation data for use in 
annotating a data file, the method comprising the steps 
of: 

receiving a typed input; 

converting words in the typed input into phoneme 
data; and 

generating annotation data by combining the phoneme 



data and words in the typed input. 

63. A method of generating annotation data for use in 
annotating a data file, the method comprising the steps 
of: 

receiving image data representative of text; 
converting said image data into text data using a 
character recognition unit; 

converting words in the text data into phoneme data; 

and 

generating annotation data by combining the phoneme 
data and words within the text data. 

64- A method according to claim 59, wherein said 
annotation data defines a phoneme and word lattice and 
wherein said generating step comprises the steps of: 

(i) generating data defining a plurality of nodes 
within the lattice and a plurality of links connecting 
the nodes within the lattice; 

(ii) generating data associating a plurality of 
phonemes of the phoneme data with a respective plurality 
of links; and 

(iii) generating data associating at least one of 
the words with at least one of said links. 

65. A method according to claim 64, wherein said 
generating step generates said data defining said phoneme 
and word lattice in blocks of said nodes. 

66. A method according to claim 64, wherein said 
generating step generates data defining time stamp 
information for each of said nodes . 

67. A method according to claim 66, wherein said 
generating step generates said phoneme and word lattice 
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data in blocks of equal time duration. 

68. A method according to claim 65, wherein said 
generating step generates data which defines each block's 
location within a database. 

69. A method according to claim 66, wherein said data 
file includes a time sequential signal, and wherein said 
generating step generates time stamp data which is time 
synchronised with said time sequential signal. 

70. A method according to claim 69, wherein said time 
sequential signal is an audio and/or video signal. 

71. A method according to claim 59, wherein said audio 
data includes audio data which defines the parol of a 
plurality of speakers, and wherein said generating step 
generates data which defines separate phoneme and word 
annotation data for the parol of each speaker. 

72. A method according to claim 60, wherein said text 
data defines the parol of a plurality of speakers, and 
wherein said generating step generates data defining 
separate phoneme and word annotation data for the parol 
of each speaker. 

73. A method according to claim 59, wherein said speech 
recognition system generates data defining a weighting 
for the phonemes associated with said links. 

74. A method according to claim 59, wherein said word 
decoder generates data defining a weighting for the words 
associated with said links. 



A method according to claim 64, wherein said step of 



defining a plurality of nodes and a plurality of links 
defines at least one node which is connected to a 
plurality of other nodes by a plurality of links. 

76. A method according to claim 75, wherein at least one 
of said plurality of links connecting said node to said 
plurality of other nodes is associated with a phoneme and 
wherein at least one of said links connecting said node 
to said plurality of other nodes is associated with a 
word. 

77. A method according to claim 61, wherein said speech 
recognition system generates data defining a weighting 
for the phonemes associated with said links. 

78. A method according to claim 61, wherein said speech 
recognition system generates data defining a weighting 
for the words associated with said links. 

79. A method according to claim 61, further comprising 
the step of associating said annotation data with said 
data file. 

80. A method according to claim 62, wherein said 
converting step uses an automatic phonetic transcription 
unit which generates said phoneme data for words within 
the typed input. 

81. A method according to claim 63, wherein said step of 
converting words into phonemes uses an automatic phonetic 
transcription unit which generates said phoneme data for 
words within the text data output by said character 
recognition unit. 



A method according to claim 63, further comprising 
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the step of associating said annotation data with either 
said received image data or with said text data. 

83. A method according to claim 63, wherein said 
receiving step uses a document scanner or a facsimile 
machine . 

84. A method of searching a data file including 
annotation data in response to an input query, the method 
comprising the steps of: 

generating phoneme data and word data corresponding 
to the input query; 

searching the data file based on the phoneme data 
and/or the word data and the annotation data; and 

outputting search results in dependence upon the 
result of said searching step. 

85. A method according to claim 84, wherein said 
annotation data defines a phoneme and word lattice 
comprising; 

(i) data for defining a plurality of nodes within 
the lattice and a plurality of links connecting the nodes 
within the lattice; 

(ii) data for associating a plurality of phonemes of 
the phoneme data with a respective plurality of links; 
and 

(iii) data for associating at least one word with at 
least one of said links. 

86. A method for storing a data file into a database, 
the method comprising the steps of: 

combining the data file with annotation data 
corresponding to the data file, the annotation data 
including phoneme data; and 

storing the data file with the annotation data. 
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87. An apparatus for searching a data file including 
annotation data, in response to an input query, the 
apparatus comprising: 

means for generating phoneme data and word data 
5 corresponding to the input query; 

means for searching a data file based on the phoneme 
data and/or the word data and the annotation data; and 

means for outputting a search result in dependence 
upon the result of said searching means. 

10 

88. An apparatus according to claim 87, wherein said 
annotation data defines a phoneme and word lattice, and 
comprises : 

(i) data defining a plurality of nodes within the 
15 lattice and a plurality of links connecting the nodes 

within the lattice; 

<ii) data associating a plurality of phonemes of the 
phoneme data with a respective plurality of links; and 
(iii) data associating at least one word with at 
20 least one of said links. 



89. An apparatus for storing a data file into a 
database, the apparatus comprising: 

means for inputting the data file and annotation 
data corresponding to the data file, the annotation data 
including phoneme data; and 

means for storing the data file with the annotation 

data. 



90. A medium for storing a data file, the data file 
comprising: 

an audio data; and 

an annotation data corresponding to the audio data, 
said annotation data including phoneme data. 
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91. A medium for storing a data file, the data file 
comprising : 

video data; 

audio data corresponding to the video data; and 
annotation data corresponding to the audio data, the 
annotation data including phoneme data. 

92. A medium for storing a data file, the data file 
comprising : 

text data; and 

annotation data corresponding to the text data, said 
annotation data including phoneme data. 

93. Data including audio data and further comprising 
annotation data corresponding to the audio data, which 
annotation data includes phoneme data. 

94. Data including video data and further comprising 
audio data corresponding to the video data and annotation 
data corresponding to the audio data, which annotation 
data includes phoneme data. 

95. Data including text data, the data further 
comprising annotation data corresponding to the text 
data, which annotation data includes phoneme data. 

96. A data carrier carrying data according to claim 1 or 
processor implementable instructions for controlling a 
processor to implement the method of claim 14 or 59 or 
84. 

97. Processor implementable instructions for controlling 
a processor to implement the method of claim 14 or 59 or 
84. 



